Managing hazards in a memory controller

ABSTRACT

Devices and techniques for managing hazards in a memory controller are described herein. The memory controller can receive a memory request that includes a base memory address. An index can be computed from the base memory address and a lookup, using the index, can be performed to find a lock. When the lock is found, the memory controller can store the memory request in a buffer that corresponds to the lock. In response to a signal to clear the lock, the memory controller removes the memory request from the buffer and performs the memory request.

PRIORITY APPLICATION

This application is a continuation of U.S. application Ser. No.17/075,365, filed Oct. 20, 2020, which is incorporated herein byreference in its entirety.

STATEMENT REGARDING GOVERNMENT SUPPORT

This invention was made with U.S. Government support under Agreement No.HR00111890003, awarded by DARPA. The U.S. Government has certain rightsin the invention.

BACKGROUND

Chiplets are an emerging technique for integrating various processingfunctionalities. Generally, a chiplet system is made up of discretemodules (each a “chiplet”) that are integrated on an interposer, and inmany examples interconnected as desired through one or more establishednetworks, to provide a system with the desired functionality. Theinterposer and included chiplets can be packaged together to facilitateinterconnection with other components of a larger system. Each chipletcan include one or more individual integrated circuits (ICs), or“chips”, potentially in combination with discrete circuit components,and commonly coupled to a respective substrate to facilitate attachmentto the interposer. Most or all chiplets in a system will be individuallyconfigured for communication through the one or more establishednetworks.

The configuration of chiplets as individual modules of a system isdistinct from such a system being implemented on single chips thatcontain distinct device blocks (e.g., intellectual property (IP) blocks)on one substrate (e.g., single die), such as a system-on-a-chip (SoC),or multiple discrete packaged devices integrated on a printed circuitboard (PCB). In general, chiplets provide better performance (e.g.,lower power consumption, reduced latency, etc.) than discrete packageddevices, and chiplets provide greater production benefits than singledie chips. These production benefits can include higher yields orreduced development costs and time.

Chiplet systems can include, for example, one or more application (orprocessor) chiplets and one or more support chiplets. Here, thedistinction between application and support chiplets is simply areference to the likely design scenarios for the chiplet system. Thus,for example, a synthetic vision chiplet system can include, by way ofexample only, an application chiplet to produce the synthetic visionoutput along with support chiplets, such as a memory controller chiplet,a sensor interface chiplet, or a communication chiplet. In a typical usecase, the synthetic vision designer can design the application chipletand source the support chiplets from other parties. Thus, the designexpenditure (e.g., in terms of time or complexity) is reduced because byavoiding the design and production of functionality embodied in thesupport chiplets. Chiplets also support the tight integration of IPblocks that can otherwise be difficult, such as those manufactured usingdifferent processing technologies or using different feature sizes (orutilizing different contact technologies or spacings). Thus, multipleIC's or IC assemblies, with different physical, electrical, orcommunication characteristics can be assembled in a modular manner toprovide an assembly providing desired functionalities. Chiplet systemscan also facilitate adaptation to suit needs of different larger systemsinto which the chiplet system will be incorporated. In an example, IC'sor other assemblies can be optimized for the power, speed, or heatgeneration for a specific function—as can happen with sensors—can beintegrated with other devices more easily than attempting to do so on asingle die. Additionally, by reducing the overall size of the die, theyield for chiplets tends to be higher than that of more complex, singledie devices.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the disclosure. The drawings, however, should not betaken to limit the disclosure to the specific embodiments, but are forexplanation and understanding only.

FIGS. 1A and 1B illustrate an example of a chiplet system, according toan embodiment.

FIG. 2 illustrates components of an example of a memory controllerchiplet, according to an embodiment.

FIG. 3 illustrates components of an example of a memory controllerchiplet, according to an embodiment.

FIG. 4 illustrates a data flow through components to support hazardmanagement, according to an embodiment.

FIG. 5 illustrates a chiplet protocol interface request packet,according to an embodiment.

FIG. 6 illustrates a chiplet protocol interface response packet,according to an embodiment.

FIG. 7 is a flow chart of an example of a method for managing hazards ina memory controller, according to an embodiment.

FIG. 8 is a block diagram of an example of a machine with which, inwhich, or by which embodiments of the present disclosure can operate.

DETAILED DESCRIPTION

FIG. 1 , described below, offers an example of a chiplet system and thecomponents operating therein. The illustrated chiplet system includes amemory controller and uses a packet-based network to communicate betweenchiplets. Details about example components and techniques implemented bythe memory controller are provided below with respect to FIGS. 1-4 .Because the chiplet memory controller or controllers that can receivepackets from any number of chiplets based on the design of the chipletsystem, an issue can arise to ensure atomicity in actions that thememory controller takes upon data when servicing requests. That is, thepotential complexity of chiplet configurations enabled by thepacket-based chiplet system described below can result in data integrityissues when, for example, traditional control structures are absent, asis often the case in chiplet systems.

To be “atomic” an action (e.g., memory update or write) is the onlyaction that can use or modify a resource at any one time. Thus, theaction exclusively modifies resources (e.g., data in memory). Thus,given two atomic actions with respect to the same resources, one actionmust begin and end before the second action can begin. Thus, atomicactions on shared resources need to be managed to ensure that the stateof the resource is being accessed by a single activity at a time.

To manage resources of the memory controller that are shared, such asmemory or computational components (e.g., cache line state, built-inatomic units, programmable atomic units, etc.) a hazard structure isused. The hazard structure leverages the fact that memory controlleractivities are invoked by memory requests to memory. Specifically, thememory requests involving the shared resources largely include at leastone memory address to which the request applies. This memory address canbe used as the basis of a control structure to determine whether aresource is being used or not.

For example, a data structure (e.g., table, array, etc.) can include avariety of entries referenced by an index (e.g., key). The value of theentry is a lock, perhaps represented as a bit (e.g., a logical oneindicates that a resource is locked and a logical zero indicates thatthe resource is unlocked). The memory address can be manipulated, suchas by hashing, to produce the index. When a request is received, thebase memory address from the request is used to determine whether thereis presently a lock in the data structure at the index derived from thebase memory address. If there is a lock, then the request is queued andtied to the lock entry—an example of this structure is illustrated inFIG. 4 . When a previous request for the same lock entry completes andclears the lock, then the oldest entry in the queue is allowed toproceed. Accordingly, the atomicity of a shared resource is managed.This is accomplished with a high throughput because a received requestthat observes a set lock entry does not stall subsequently receivedrequests and ordering of received requests hashed to the individual lockentries is maintained. Additional details and examples are providedbelow.

FIGS. 1A and 1B illustrate an example of a chiplet system 110, accordingto an embodiment. FIG. 1A is a representation of the chiplet system 110mounted on a peripheral board 105, that can be connected to a broadercomputer system by a peripheral component interconnect express (PCIe),for example. The chiplet system 110 includes a package substrate 115, aninterposer 120, and four chiplets, an application chiplet 125, a hostinterface chiplet 135, a memory controller chiplet 140, and a memorydevice chiplet 150. Other systems can include many additional chipletsto provide additional functionalities as will be apparent from thefollowing discussion. The package of the chiplet system 110 isillustrated with a lid or cover 165, though other packaging techniquesand structures for the chiplet system can be used. FIG. 1B is a blockdiagram labeling the components in the chiplet system for clarity.

The application chiplet 125 is illustrated as including anetwork-on-chip (NOC) 130 to support a chiplet network 155 forinter-chiplet communications. In example embodiments NOC 130 can beincluded on the application chiplet 125. In an example, NOC 130 can bedefined in response to selected support chiplets (e.g., chiplets 135,140, and 150) thus enabling a designer to select an appropriate numberor chiplet network connections or switches for the NOC 130. In anexample, the NOC 130 can be located on a separate chiplet, or evenwithin the interposer 120. In examples as discussed herein, the NOC 130implements a chiplet protocol interface (CPI) network.

The CPI is a packet-based network that supports virtual channels toenable a flexible and high-speed interaction between chiplets. CPIenables bridging from intra-chiplet networks to the chiplet network 155.For example, the Advanced eXtensible Interface (AXI) is a widely usedspecification to design intra-chip communications. AXI specifications,however, cover a great variety of physical design options, such as thenumber of physical channels, signal timing, power, etc. Within a singlechip, these options are generally selected to meet design goals, such aspower consumption, speed, etc. However, to achieve the flexibility ofthe chiplet system, an adapter, such as CPI, is used to interfacebetween the various AXI design options that can be implemented in thevarious chiplets. By enabling a physical channel to virtual channelmapping and encapsulating time-based signaling with a packetizedprotocol, CPI bridges intra-chiplet networks across the chiplet network155.

CPI can use a variety of different physical layers to transmit packets.The physical layer can include simple conductive connections, or caninclude drivers to increase the voltage, or otherwise facilitatetransmitting the signals over longer distances. An example of one suchphysical layer can include the Advanced Interface Bus (AIB), which invarious examples, can be implemented in the interposer 120. AIBtransmits and receives data using source synchronous data transfers witha forwarded clock. Packets are transferred across the AIB at single datarate (SDR) or dual data rate (DDR) with respect to the transmittedclock. Various channel widths are supported by AIB. AIB channel widthsare in multiples of 20 bits when operated in SDR mode (20, 40, 60, . . .), and multiples of 40 bits for DDR mode: (40, 80, 120, . . . ). The AIBchannel width includes both transmit and receive signals. The channelcan be configured to have a symmetrical number of transmit (TX) andreceive (RX) input/outputs (I/Os), or have a non-symmetrical number oftransmitters and receivers (e.g., either all transmitters or allreceivers). The channel can act as an AIB principal or subordinatedepending on which chiplet provides the principal clock. AIB I/O cellssupport three clocking modes: asynchronous (i.e. non-clocked), SDR, andDDR. In various examples, the non-clocked mode is used for clocks andsome control signals. The SDR mode can use dedicated SDR only I/O cells,or dual use SDR/DDR I/O cells.

In an example, CPI packet protocols (e.g., point-to-point or routable)can use symmetrical receive and transmit I/O cells within an AIBchannel. The CPI streaming protocol allows more flexible use of the AIBI/O cells. In an example, an AIB channel for streaming mode canconfigure the I/O cells as all TX, all RX, or half TX and half RX. CPIpacket protocols can use an AIB channel in either SDR or DDR operationmodes. In an example, the AIB channel is configured in increments of 80I/O cells (i.e. 40 TX and 40 RX) for SDR mode and 40 I/O cells for DDRmode. The CPI streaming protocol can use an AIB channel in either SDR orDDR operation modes. Here, in an example, the AIB channel is inincrements of 40 I/O cells for both SDR and DDR modes. In an example,each AIB channel is assigned a unique interface identifier. Theidentifier is used during CPI reset and initialization to determinepaired AIB channels across adjacent chiplets. In an example, theinterface identifier is a 20-bit value comprising a seven-bit chipletidentifier, a seven-bit column identifier, and a six-bit linkidentifier. The AIB physical layer transmits the interface identifierusing an AIB out-of-band shift register. The 20-bit interface identifieris transferred in both directions across an AIB interface using bits32-51 of the shift registers.

AIB defines a stacked set of AIB channels as an AIB channel column. AnAIB channel column has some number of AIB channels, plus an auxiliarychannel. The auxiliary channel contains signals used for AIBinitialization. All AIB channels (other than the auxiliary channel)within a column are of the same configuration (e.g., all TX, all RX, orhalf TX and half RX, as well as having the same number of data I/Osignals). In an example, AIB channels are numbered in continuousincreasing order starting with the AIB channel adjacent to the AUXchannel. The AIB channel adjacent to the AUX is defined to be AIBchannel zero.

Generally, CPI interfaces on individual chiplets can includeserialization-deserialization (SERDES) hardware. SERDES interconnectswork well for scenarios in which high-speed signaling with low signalcount are desirable. SERDES, however, can result in additional powerconsumption and longer latencies for multiplexing and demultiplexing,error detection or correction (e.g., using block level cyclic redundancychecking (CRC)), link-level retry, or forward error correction. However,when low latency or energy consumption is a primary concern forultra-short reach, chiplet-to-chiplet interconnects, a parallelinterface with clock rates that allow data transfer with minimal latencycan be utilized. CPI includes elements to minimize both latency andenergy consumption in these ultra-short reach chiplet interconnects.

For flow control, CPI employs a credit-based technique. A recipient,such as the application chiplet 125, provides a sender, such as thememory controller chiplet 140, with credits that represent availablebuffers. In an example, a CPI recipient includes a buffer for eachvirtual channel for a given time-unit of transmission. Thus, if the CPIrecipient supports five messages in time and a single virtual channel,the recipient has five buffers arranged in five rows (e.g., one row foreach unit time). If four virtual channels are supported, then therecipient has twenty buffers arranged in five rows. Each buffer holdsthe payload of one CPI packet.

When the sender transmits to the recipient, the sender decrements theavailable credits based on the transmission. Once all credits for therecipient are consumed, the sender stops sending packets to therecipient. This ensures that the recipient always has an availablebuffer to store the transmission.

As the recipient processes received packets and frees buffers, therecipient communicates the available buffer space back to the sender.This credit return can then be used by the sender allow transmitting ofadditional information.

Also illustrated is a chiplet mesh network 160 that uses a direct,chiplet-to-chiplet technique without the need for the NOC 130. Thechiplet mesh network 160 can be implemented in CPI, or anotherchiplet-to-chiplet protocol. The chiplet mesh network 160 generallyenables a pipeline of chiplets where one chiplet serves as the interfaceto the pipeline while other chiplets in the pipeline interface only withthemselves.

Additionally, dedicated device interfaces, such as one or more industrystandard memory interfaces 145 (such as, for example, synchronous memoryinterfaces, such as DDR5, DDR6), can also be used to interconnectchiplets. Connection of a chiplet system or individual chiplets toexternal devices (such as a larger system can be through a desiredinterface (for example, a PCIE interface). Such as external interfacecan be implemented, in an example, through a host interface chiplet 135,which in the depicted example, provides a PCIE interface external tochiplet system 110. Such dedicated interfaces 145 are generally employedwhen a convention or standard in the industry has converged on such aninterface. The illustrated example of a Double Data Rate (DDR) interface145 connecting the memory controller chiplet 140 to a dynamic randomaccess memory (DRAM) memory device 150 is just such an industryconvention.

Of the variety of possible support chiplets, the memory controllerchiplet 140 is likely present in the chiplet system 110 due to the nearomnipresent use of storage for computer processing as well assophisticated state-of-the-art for memory devices. Thus, using memorydevice chiplets 150 and memory controller chiplets 140 produced byothers gives chiplet system designers access to robust products bysophisticated producers. Generally, the memory controller chiplet 140provides a memory device specific interface to read, write, or erasedata. Often, the memory controller chiplet 140 can provide additionalfeatures, such as error detection, error correction, maintenanceoperations, or atomic operator execution. For some types of memory,maintenance operations tend to be specific to the memory device 150,such as garbage collection in NAND flash or storage class memories,temperature adjustments (e.g., cross temperature management) in NANDflash memories. In an example, the maintenance operations can includelogical-to-physical (L2P) mapping or management to provide a level ofindirection between the physical and logical representation of data. Inother types of memory, for example DRAM, some memory operations, such asrefresh can be controlled by a host processor or of a memory controllerat some times, and at other times controlled by the DRAM memory device,or by logic associated with one or more DRAM devices, such as aninterface chip (in an example, a buffer).

Atomic operators are a data manipulation that, for example, can beperformed by the memory controller chiplet 140. In other chipletsystems, the atomic operators can be performed by other chiplets. Forexample, an atomic operator of “increment” can be specified in a commandby the application chiplet 125, the command including a memory addressand possibly an increment value. Upon receiving the command, the memorycontroller chiplet 140 retrieves a number from the specified memoryaddress, increments the number by the amount specified in the command,and stores the result. Upon a successful completion, the memorycontroller chiplet 140 provides an indication of the commands success tothe application chiplet 125. Atomic operators avoid transmitting thedata across the chiplet network 160, resulting in lower latencyexecution of such commands.

Atomic operators can be classified as built-in atomics or programmable(e.g., custom) atomics. Built-in atomics are a finite set of operationsthat are immutably implemented in hardware. Programmable atomics aresmall programs that can execute on a programmable atomic unit (PAU)(e.g., a custom atomic unit (CAU)) of the memory controller chiplet 140.FIG. 1 illustrates an example of a memory controller chiplet thatdiscusses a PAU.

The memory device chiplet 150 can be, or include any combination of,volatile memory devices or non-volatile memories. Examples of volatilememory devices include, but are not limited to, random access memory(RAM)—such as DRAM) synchronous DRAM (SDRAM), graphics double data ratetype 6 SDRAM (GDDR6 SDRAM), among others. Examples of non-volatilememory devices include, but are not limited to, negative-and-(NAND)—typeflash memory, storage class memory (e.g., phase-change memory ormemristor based technologies), ferroelectric RAM (FeRAM), among others.The illustrated example includes the memory device 150 as a chiplet,however, the memory device 150 can reside elsewhere, such as in adifferent package on the peripheral board 105. For many applications,multiple memory device chiplets can be provided. In an example, thesememory device chiplets can each implement one or multiple storagetechnologies. In an example, a memory chiplet can include, multiplestacked memory die of different technologies, for example one or morestatic random access memory (SRAM) devices stacked or otherwise incommunication with one or more dynamic random access memory (DRAM)devices. Memory controller 140 can also serve to coordinate operationsbetween multiple memory chiplets in chiplet system 110; for example, toutilize one or more memory chiplets in one or more levels of cachestorage, and to use one or more additional memory chiplets as mainmemory. Chiplet system 110 can also include multiple memory controllers140, as can be used to provide memory control functionality for separateprocessors, sensors, networks, etc. A chiplet architecture, such aschiplet system 110 offers advantages in allowing adaptation to differentmemory storage technologies; and different memory interfaces, throughupdated chiplet configurations, without requiring redesign of theremainder of the system structure.

FIG. 2 illustrates components of an example of a memory controllerchiplet 205, according to an embodiment. The memory controller chiplet205 includes a cache 210, a cache controller 215, an off-die memorycontroller 220 (e.g., to communicate with off-die memory 275), a networkcommunication interface 225 (e.g., to interface with a chiplet network285 and communicate with other chiplets), and a set of atomic and mergeunits 250. Members of this set can include, for example, a write mergeunit 255, a memory hazard unit 260, built-in atomic unit 265, or a PAU270. The various components are illustrated logically, and not as theynecessarily would be implemented. For example, the built-in atomic unit265 likely comprises different devices along a path to the off-diememory. For example, the built-in atomic unit 265 could be in aninterface device/buffer on a memory chiplet, as discussed above. Incontrast, the programmable atomic unit 270 could be implemented in aseparate processor on the memory controller chiplet 205 (but in variousexamples can be implemented in other locations, for example on a memorychiplet).

The off-die memory controller 220 is directly coupled to the off-diememory 275 (e.g., via a bus or other communication connection) toprovide write operations and read operations to and from the one or moreoff-die memory, such as off-die memory 275 and off-die memory 280. Inthe depicted example, the off-die memory controller 220 is also coupledfor output to the atomic and merge unit 250, and for input to the cachecontroller 215 (e.g., a memory side cache controller).

In the example configuration, cache controller 215 is directly coupledto the cache 210, and can be coupled to the network communicationinterface 225 for input (such as incoming read or write requests), andcoupled for output to the off-die memory controller 220.

The network communication interface 225 includes a packet decoder 230,network input queues 235, a packet encoder 240, and network outputqueues 245 to support a packet-based chiplet network 285, such as CPI.The chiplet network 285 can provide packet routing between and amongprocessors, memory controllers, hybrid threading processors,configurable processing circuits, or communication interfaces. In such apacket-based communication system, each packet typically includesdestination and source addressing, along with any data payload orinstruction. In an example, the chiplet network 285 can be implementedas a collection of crossbar switches having a folded Clos configuration,or a mesh network providing for additional connections, depending uponthe configuration.

In various examples, the chiplet network 285 can be part of anasynchronous switching fabric. Here, a data packet can be routed alongany of various paths, such that the arrival of any selected data packetat an addressed destination can occur at any of multiple differenttimes, depending upon the routing. Additionally, chiplet network 285 canbe implemented at least in part as a synchronous communication network,such as a synchronous mesh communication network. Both configurations ofcommunication networks are contemplated for use for examples inaccordance with the present disclosure.

The memory controller chiplet 205 can receive a packet having, forexample, a source address, a read request, and a physical address. Inresponse, the off-die memory controller 220 or the cache controller 215will read the data from the specified physical address (which can be inthe off-die memory 275 or in the cache 210), and assemble a responsepacket to the source address containing the requested data. Similarly,the memory controller chiplet 205 can receive a packet having a sourceaddress, a write request, and a physical address. In response, thememory controller chiplet 205 will write the data to the specifiedphysical address (which can be in the cache 210 or in the off-diememories 275 or 280), and assemble a response packet to the sourceaddress containing an acknowledgement that the data was stored to amemory.

Thus, the memory controller chiplet 205 can receive read and writerequests via the chiplet network 285 and process the requests using thecache controller 215 interfacing with the cache 210, if possible. If therequest cannot be handled by the cache controller 215, the off-diememory controller 220 handles the request by communication with theoff-die memories 275 or 280, the atomic and merge unit 250, or both. Asnoted above, one or more levels of cache can also be implemented inoff-die memories 275 or 280; and in some such examples can be accesseddirectly by cache controller 215. Data read by the off-die memorycontroller 220 can be cached in the cache 210 by the cache controller215 for later use.

The atomic and merge unit 250 are coupled to receive (as input) theoutput of the off-die memory controller 220, and to provide output tothe cache 210, the network communication interface 225, or directly tothe chiplet network 285. The memory hazard unit 260, write merge unit255 and the built-in (e.g., predetermined) atomic unit 265 can each beimplemented as state machines with other combinational logic circuitry(such as adders, shifters, comparators, AND gates, OR gates, XOR gates,or any suitable combination thereof) or other logic circuitry. Thesecomponents can also include one or more registers or buffers to storeoperand or other data. The PAU 270 can be implemented as one or moreprocessor cores or control circuitry, and various state machines withother combinational logic circuitry or other logic circuitry, and canalso include one or more registers, buffers, or memories to storeaddresses, executable instructions, operand and other data, or can beimplemented as a processor.

The write merge unit 255 receives read data and request data, and mergesthe request data and read data to create a single unit having the readdata and the source address to be used in the response or return datapacket). The write merge unit 255 provides the merged data to the writeport of the cache 210 (or, equivalently, to the cache controller 215 towrite to the cache 210). Optionally, the write merge unit 255 providesthe merged data to the network communication interface 225 to encode andprepare a response or return data packet for transmission on the chipletnetwork 285.

When the request data is for a built-in atomic operator, the built-inatomic unit 265 receives the request and reads data, either from thewrite merge unit 255 or directly from the off-die memory controller 220.The atomic operator is performed, and using the write merge unit 255,the resulting data is written to the cache 210, or provided to thenetwork communication interface 225 to encode and prepare a response orreturn data packet for transmission on the chiplet network 285.

The built-in atomic unit 265 handles predefined atomic operators such asfetch-and-increment or compare-and-swap. In an example, these operationsperform a simple read-modify-write operation to a single memory locationof 32-bytes or less in size. Atomic memory operations are initiated froma request packet transmitted over the chiplet network 285. The requestpacket has a physical address, atomic operator type, operand size, andoptionally up to 32-bytes of data. The atomic operator performs theread-modify-write to a cache memory line of the cache 210, filling thecache memory if necessary. The atomic operator response can be a simplecompletion response, or a response with up to 32-bytes of data. Exampleatomic memory operators include fetch-and-AND, fetch-and-OR,fetch-and-XOR, fetch-and-add, fetch-and-subtract, fetch-and-increment,fetch-and-decrement, fetch-and-minimum, fetch-and-maximum,fetch-and-swap, and compare-and-swap. In various example embodiments,32-bit and 64-bit operations are supported, along with operations on 16or 32 bytes of data. Methods disclosed herein are also compatible withhardware supporting larger or smaller operations and more or less data.

Built-in atomic operators can also involve requests for a “standard”atomic operator on the requested data, such as comparatively simple,single cycle, integer atomics—such as fetch-and-increment orcompare-and-swap—which will occur with the same throughput as a regularmemory read or write operation not involving an atomic operator. Forthese operations, the cache controller 215 can generally reserve a cacheline in the cache 210 by setting a hazard (in hardware), so that thecache line cannot be read by another process while it is in transition.An example of components and operations of the hazard mechanism canoperate as described below with respect to FIGS. 3 (e.g., the Hazardblock) and 4. The data is obtained from either the off-die memory 275 orthe cache 210, and is provided to the built-in atomic unit 265 toperform the requested atomic operator. Following the atomic operator, inaddition to providing the resulting data to the packet encoder 240 toencode outgoing data packets for transmission on the chiplet network285, the built-in atomic unit 265 provides the resulting data to thewrite merge unit 255, which will also write the resulting data to thecache 210. Following the writing of the resulting data to the cache 210,any corresponding hazard which was set will be cleared by the memoryhazard unit 260.

The PAU 270 enables high performance (high throughput and low latency)for programmable atomic operators (also referred to as “custom atomictransactions” or “custom atomic operators”), comparable to theperformance of built-in atomic operators. Rather than executing multiplememory accesses, in response to an atomic operator request designating aprogrammable atomic operator and a memory address, circuitry in thememory controller chiplet 205 transfers the atomic operator request toPAU 270 and sets a hazard stored in a memory hazard register, forexample, that corresponds to the memory address of the memory line usedin the atomic operator to ensure that no other operation (read, write,or atomic) is performed on that memory line. The hazard is cleared uponcompletion of the atomic operator. Again, an example of components andoperations of the hazard mechanism can operate as described below withrespect to FIGS. 3 (e.g., the Hazard block) and 4. Additional, directdata paths provided for the PAU 270 executing the programmable atomicoperators allow for additional write operations without any limitationsimposed by the bandwidth of the communication networks and withoutincreasing any congestion of the communication networks.

The PAU 270 includes a multi-threaded processor, for example, such as aRISC-V ISA based multi-threaded processor, having one or more processorcores, and further having an extended instruction set for executingprogrammable atomic operators. When provided with the extendedinstruction set for executing programmable atomic operators, the PAU 270can be embodied as one or more hybrid threading processors. In someexample embodiments, the PAU 270 provides barrel-style, round-robininstantaneous thread switching to maintain a high instruction-per-clockrate.

Programmable atomic operators can be performed by the PAU 270 involvingrequests for a programmable atomic operator on the requested data. Auser can prepare programming code to provide such programmable atomicoperators. For example, the programmable atomic operators can becomparatively simple, multi-cycle operations such as floating-pointaddition, or comparatively complex, multi-instruction operations such asa Bloom filter insert. The programmable atomic operators can be the sameas or different than the predetermined atomic operators, insofar as theyare defined by the user rather than a system vendor. For theseoperations, the cache controller 215 can reserve a cache line in thecache 210, by setting a hazard, so that cache line cannot be read byanother process while it is in transition. The data is obtained fromeither the cache 210 or the off-die memories 275 or 280, and is providedto the PAU 270 to perform the requested programmable atomic operator.Following the atomic operator, the PAU 270 will provide the resultingdata to the network communication interface 225 to directly encodeoutgoing data packets having the resulting data for transmission on thechiplet network 285. In addition, the PAU 270 will provide the resultingdata to the cache controller 215, which will also write the resultingdata to the cache 210. Following the writing of the resulting data tothe cache 210, any corresponding hazard which was set will be cleared bythe cache control circuit 215.

In selected examples, the approach taken for programmable atomicoperators is to provide multiple, generic, custom atomic request typesthat can be sent through the chiplet network 285 to the memorycontroller chiplet 205 from an originating source such as a processor orother system component. The cache controllers 215 or off-die memorycontroller 220 identify the request as a custom atomic and forward therequest to the PAU 270. In a representative embodiment, the PAU 270: (1)is a programmable processing element capable of efficiently performing auser defined atomic operator; (2) can perform load and stores to memory,arithmetic and logical operations and control flow decisions; and (3)leverages the RISC-V ISA with a set of new, specialized instructions tofacilitate interacting with such controllers 215, 220 to atomicallyperform the user-defined operation. In desirable examples, the RISC-VISA contains a full set of instructions that support high level languageoperators and data types. The PAU 270 can leverage the RISC-V ISA, butwill commonly support a more limited set of instructions and limitedregister file size to reduce the die size of the unit when includedwithin the memory controller chiplet 205.

As mentioned above, prior to the writing of the read data to the cache210, the hazard is set for the reserved cache line is to be cleared, bythe memory hazard clear unit 260 (e.g., Hazard Clear block 365).Accordingly, when the request and read data is received by the writemerge unit 255, a reset or clear signal can be transmitted by the memoryhazard clear unit 260 to the cache 210 to reset the set memory hazardfor the reserved cache line. Also, resetting this hazard will alsorelease a pending read or write request involving the designated (orreserved) cache line, providing the pending read or write request to aninbound request multiplexer for selection and processing.

FIG. 3 illustrates components of an example of a memory controllerchiplet, according to an embodiment. FIG. 3 is another representation ofa memory controller from the memory controller 205 illustrated in FIG. 2. Many of the same components shown in FIG. 2 are illustrated here. Forexample, the cache 302 and 385 are examples of cache 210; DRAM(s) 340are examples of off-die memory 275-280; atomic/write merge 370 and theprogrammable atomic unit 380 may be an example of atomics and merge unit250. Other components of FIG. 3 may be examples of other components ofFIG. 2 such as off-die memory controller 220 and cache controller 215.

Other components, not specifically represented in the memory controller205, can include the following. A NOC Request Queue 305 to receiverequests from the network-on-chip and provide a small amount of queuing.An Atomic Request Queue 310 that receives requests from the programmableatomic unit 380 and provides a small amount of queuing. An InboundRequest Multiplexer (IRM) that selects between inbound memory requestsources. In an example, the three memory request sources, in order ofpriority are: Memory Hazard Requests, Atomic Requests, and Inbound NOCRequests.

The Cache (Read) 325 and Cache (Write) 375 are a single deviceimplemented as, in an example, an SRAM data cache. The diagramillustrates the cache as two separate blocks (325 and 375), oneproviding read access, the other providing write access. A Delay block320 provides one or more pipeline stages to mimic the delay for an SRAMcache read operation. Generally, a cache miss accesses to the off-diememory 340 (e.g., off-die memory 280) to bring the desired data into thecache. While waiting for the memory response (e.g., access time for theDRAM 340), the memory line is not available for other requests.

A Memory Hazard block (Hazard Set block 315 and Hazard Clear block 365)can maintain a table of hazards indicating which shared resources of thememory controller—such as memory lines, cache lines, PAU operations,etc.)—are free or unavailable because they are already in use. Thus, theMemory Hazard block operates to ensure atomicity of requests thoughexclusive access to shared resources. For example, an inbound requestthat tries to access a shared resource with a hazard is held by theMemory Hazard block until the hazard is cleared. Once the hazard iscleared then the request can proceed (e.g., the request can be resentthrough the Inbound Request Multiplexer). In an example, the hazards forresources are based on (e.g., identified by, linked to, etc.) a memoryaddress present in the request. Such an addressed-based hazard mechanismleverages the presence of the memory address in requests to the memorycontroller and the usual correspondence of such physical memoryaddresses and hardware within the memory controller.

The Memory Hazard block (e.g., Hazard Set block 315) is configured toreceive a memory request. This memory request includes a base memoryaddress. Here, the term “base memory address” refers to one memoryaddress in the memory request if more than one memory address isspecified. The selection of the one memory address is a design decision,but the selection is not variable. Thus, if the base memory address isthe lower bound of a range specified by two memory address in the memoryrequest, then the base memory address is always the lower bound. In anexample, the memory request is for data in a memory managed by a memorycontroller. The issue is consistency, such that the single base memoryaddress is the same across multiple memory requests for the same sharedresources.

In an example, the request is received via the NOC as a CPI request,such as the CPI request 500 described below with respect to FIG. 5 . Inan example, the memory request is for a computation component, such as acryptographic unit (not shown), a built-in atomic unit (e.g., built-inatomic unit 265), or the PAU 380. In an example, the memory request isin the form of a CPI request, such as the CPI request 500 of FIG. 5 .

The Memory Hazard block is configured to compute an index from the basememory address. In an example, the index is computed from the basememory address by hashing the base memory address. Hashing the basememory address converts the base memory address into a consistentrepresentation that is generally more condensed that the base memoryaddress. The hashing technique, such as the number of output bits thatare produced, can be adjusted based on contention for resources.Generally, the fewer bits in the output hash, the more likely that acollision occurs. Here a collision occurs when two different memoryaddresses are hashed and produce the same output. Thus, when contentionis low, fewer bits can be used for the hash resulting in greater economyof hardware to support the mechanism. Conversely, when contention ishigh, there is a greater likelihood of a collision and more bits can beused for the hash.

The Memory Hazard block is configured to perform a lookup using theindex to find a lock. Here, the lookup is performed against a datastructure with entries. The value in the entry indicates whether or notthere is a lock and the index is used to uniquely identify an entry.Thus, in an example, to perform the lookup using the index, the entry inthe data structure that corresponds to the index is obtained (e.g.,retrieved). In an example, the data structure is an array. In thisexample, the base memory address is hashed to an integer number thatcorresponds to an entry of the array. Thus, the memory address is hashedto an index of the array. In this example, the lock value is held in thearray at the index.

In an example, the data structure has fewer entries than base memoryaddresses in an addressable space for the memory controller. Thisexample demonstrates the power in using a hash of the base memoryaddress rather than the base memory address itself. Generally, it isunlikely that all memory addresses in the memory controller will besubject to simultaneous requests. If the memory address were useddirectly, the data structure would likely maintain many entries that arefrequently unused. This results in wasted hardware resources dedicatedto maintaining lock values that are rarely consulted. In this example, afewer number of data structure entries are used than all possible memoryaddresses, reducing the amount of hardware used to locks. As notedabove, the number of bits of the hash—and thus the number of datastructure entries—can be tailored to control collisions. Generally, thesmaller the lock value data structure, the greater chance that any twomemory requests will collide. Thus, a smaller data structure is hardwareefficient but can result in greater collisions (e.g., contention) andreduce throughput.

In an example, the lock is represented by a single bit, such as alogical one meaning that the shared resource is locked and a logicalzero meaning that the resources are unlocked. While using a single bitfor the lock can be efficient, other techniques can also be used. Forexample, as noted below and illustrated in FIG. 4 , a buffer (e.g., aqueue) can be linked to the lock to hold requests waiting for the sharedresource to be free. Here, the mere presence of requests in the buffercan be used as the lock value, the buffer being the entry in the datastructure and identified by the hash, if the request at the head of thebuffer is not removed until after it completes.

The Memory Hazard block is configured to store the memory request in abuffer in response to finding the lock. Thus, when the memory address isconverted into the index, and the index is used to retrieve an entry inthe data structure, the entry indicates whether or not there is a lock.When there is a lock, the memory request cannot proceed because theshared resource is already in use. The memory request is then held inthe buffer to await the completion of the operation that is using theshared resource. Storing the memory request in the buffer means thatwhatever portion of the memory request that would have been passed intothe scheduler 335, for example, is stored. Thus, although the memoryrequest upon receipt at the memory controller can include packetinformation to transit a chiplet network, if that information is notneeded to perform the memory request, then it can be excluded from thebuffer. Similarly, storing the memory request into the buffer caninclude storing a key for the memory request, where the data of thememory request is held at another component (e.g., a different buffer)in the memory controller. This technique can reduce the amount of datathat is copied, for example, as the memory request moves through thebuffer. In any example, the storage of the memory request maintainsorder between different memory requests, arriving at different times(with possibly different quality of server (QoS) values).

In an example, the buffer is one of several buffers. Here, the buffer towhich the memory request is stored is the only one of the severalbuffers that corresponds to the lock and the other buffers respectivelycorrespond to other possible locks. Thus, each buffer uniquelycorresponds to one possible lock, linking the buffers and the possiblelocks. While a single buffer could be used, additional meta data wouldlikely be needed to ensure that a next memory request for a freed sharedresource does not wait on another memory request waiting for a stilllocked shared resource. Separate buffers for each index in the datastructure provides a straightforward technique to avoid such issues.

The Memory Hazard block is configured to remove the memory request fromthe buffer when a signal to clear the lock is received. This signal caninclude an indication (e.g., interrupt, state register setting, etc.)that the memory request that caused the lock has completed, actualsetting of the lock (e.g., from a logical one to a logical zero), arequest to clear the lock, among other things. Thus, when the lock iscleared, the next memory request in the buffer can be removed andperformed by the memory controller.

In an example, removing the memory request from the buffer includesselecting the buffer from multiple of the several buffers by an arbiter.This example contemplates the simultaneous (e.g., on a same clock cycleor during a window of time such as a scheduling interval) freeing ofmultiple locks upon which memory requests are waiting. Thus, the arbiteruses a technique, such as a round-robin selection, to choose which ofthe buffered memory requests will be removed and passed down theprocessing pipeline for the memory controller. In an example, selectionsof the arbiter are placed in a queue. This queue is an output queue fromwhich the memory controller pipeline can select a next waiting requestto process. In an example, the memory request is removed from the queuein response to an absence of a memory request that had no lock. Thisexample notes a design choice in which a non-block memory request isprocessed with priority over a block memory request during a givenscheduling interval. Although this can possibly lead to increasedlatencies for memory requests with contention, this arrangementeliminates the need to buffer memory requests not subject to a lock. Anexample of this arrangement is illustrated in FIG. 4 .

The memory controller is configured to perform the memory request afterremoval from the buffer. Here, the memory request has a right to theshared resource over other memory requests. Thus, once removed from thebuffer, the memory request proceeds down the memory controllerprocessing pipeline to completion.

In an example, the Memory Hazard block is configured to set the lock aspart of performing the memory request. Thus, as the memory requestproceeds down the processing pipeline, the lock is set to ensure that nofollowing memory request for the same shared resources is scheduleduntil the memory request completes. Setting the lock value generallyoccurs before the memory request is passed down the pipeline to ensurethat there are no race conditions. Accordingly, as soon as the lockvalue is verified to indicate that there is no lock, the lock value isset on behalf of the memory request.

An Inbound DRAM Control Multiplexer (IDCM) selects from an inbound NOCrequest and a cache eviction request. For the Bank Request Queues 330,each separately managed DRAM bank has a dedicated bank request queue tohold requests until they can be scheduled on the associated DRAM bank.

The scheduler 335 selects across the bank request queues 335 to choose arequest for an available DRAM bank. A Request Hit Data Queue 360 holdsrequest data from cache hits until selected. A Request Miss Data Queue355 holds data read from the DRAM(s) until selected. A Miss RequestQueue 350 is used to hold request packet information for cache missesuntil the request is selected. A Hit Request Queue 345 holds requestpacket information for cache hits until selected. A Data SelectionMultiplexer (DSM) selects between DRAM read data and cache hit readdata. The selected data is written to the SRAM cache. Request SelectionMultiplexer (RSM) selects between hit and miss request queues 345 and355.

The Atomic/Write Merge 370 either merges the request data and DRAM readdata, or, if the request is a built-in atomic (e.g., built-in atomicoperation block 265), the memory data and request data are used asinputs for an atomic operation. The Cache (Write) block 375 representsthe write port for the SRAM cache. Data from a NOC write request anddata from DRAM read operations are written to the SRAM cache. The MemoryHazard (Clear) block 365 represents the hazard clear operation for thememory hazard structure. Clearing a hazard may release a pending NOCrequest and send it to the Inbound Request Multiplexer. The programmableAtomic Unit 380 processes programmable atomic operations (e.g.,transactions). The NOC Outbound Response Multiplexer (ORM) selectsbetween memory controller responses and custom atomic unit responses andsends the selection to the NOC.

FIG. 4 illustrates a data flow through components to support hazardmanagement, according to an embodiment. These components can includecircuitry to perform a hash (hash block 405) on an address in a memoryrequest. In an example, the hash block 405 can be replaced orsupplemented by other computation units that transform the address inthe memory request to a suitable form for the lock table 410,collectively referred to a transform circuitry. The components alsoinclude a lock table 410 data structure, a linked queue 415 datastructure, an arbiter 420, an output queue 425 data structure, and anoutput multiplexer 430. The data structures are implemented incircuitry, such as by a RAM device, flip-flop registers, etc. Thesecomponents can be included in any device for which address-basedresource locking is desired, such as a memory controller (e.g., memorycontroller 140 of FIG. 1 , memory controller 205 of FIG. 2 , Hazardblock in the memory controller of FIG. 3 , etc.) or a PAU (e.g., PAU 270of FIG. 2 , PAU 380 of FIG. 3 , etc.), among others.

As illustrated, the address (e.g., base memory address) of the requestis provided to a hash unit 405 to compute and index. The index uniquelycorresponds to an entry in the lock table 410. If there is no lock atthe entry, the lock is set at the entry and the request proceeds to themultiplexer 430 and then down the processing pipeline. Here, the requestfor which no lock is present upon the request's arrival is givenpriority by the multiplexer.

If there is a lock at the entry in the lock table 410, then the requestis buffered in the queue 415 that corresponds to the lock table entry(and hence the index from the hash unit 405). The queue 415 can beimplemented as a first-in-first-out (FIFO), ensuring that arrival orderof buffered memory requests is maintained. When a response signals thatthe lock corresponding to the index of the base memory address of therequest that resulted in the response should be cleared, the head of thequeue 415 corresponding to that index can be removed by the arbiter 420.In an example, lock entry can be modified (e.g., cleared) to provoke thedequeuing of the next memory request. In an example, the signal canprovoke the dequeuing without modifying the lock entry.

The arbiter 420 addresses situations in which multiple locks arecleared, and thus multiple memory requests can be dequeued. Here, thearbiter selects from the multiple available memory requests. Thisselection can take many forms, such as a round-robin selection of thequeues.

The arbiter 420 provides the selected memory request to an output queue425. At each scheduling interval, the multiplexer 430 removes a nextmemory request from the output queue 425 when there are no passthroughmemory requests. Again, this configuration of the multiplexerprioritizes the passthrough, or contention free, memory request overmemory requests that had contention on a shared resource when theyarrived.

FIG. 5 illustrates a chiplet protocol interface request packet 500,according to an embodiment. The following is a table for an example ofCPI field descriptions and bit lengths corresponding to the CPI requestpacket 500.

Field Field Name Width Value Field Description Line 1 CMD 8 126 Extendedvirtual channel 1 (VC1) LEN 5 Packet Length SC 1 0 Sequence Continue(ignored for external memory device (EMD)) DID 12 Destination NOCendpoint PATH 8 Endpoint Offset <14:7> CP 2 1 Credit/Path Order (CreditReturn enabled in flits 3-N and PATH field based path ordering) Line 2TU 2 Transaction ID <9:8> EPOff 7 Endpoint Offset <6:0> <6:0> TA 8Transaction IS <7:0> EpOffset 19 Endpoint Offset <33:15> <33:15> Line 3EXCMD 8 Extended Command BTYPE 4 8 BTYPE of 8 is EMD vendor defined SID12 Source NOC Endpoint EpOffset 4 Endpoint Offset <37:34> <37:34> RSV 40 Reserved CR/RSV 4 Credit Return Line 4 CrPKnd 4 Credit Pool KindCrPIdx 8 Credit Pool Index RSV 4 0 Reserved CaPIdx 8 Custom(Programmable) Atomic Partition Index Calntv 8 Interleave Size CR/RSV 4Credit Return Lines 5 and Beyond DATA 32 Argument data: 0, 1, 2, or 4,64-bit values CR/RSV 4 Credit Return

As illustrated, line 4, the shaded line is an extended header 510. Thecommand field 505 indicates that the request 500 is for a PAO. However,the entity decoding the request 500 and providing the PAO parameter to aPAU (e.g., PAU 270) will either pass the extended header 510 informationto the PAU or decode the extended header 510 and provide the constituentfields as inputs to the PAU.

FIG. 6 illustrates a chiplet protocol interface response packet 600,according to an embodiment. The following is a table for an example ofCPI field descriptions and bit lengths corresponding to the CPI responsepacket 600.

Field Field Name Width Field Description Line 1 CMD 8 Packet command LEN5 Encoded packet Length SC 1 Sequence Continue. When set, this packet ispart of a multi-packet transfer and this packet is not the last packetin the sequence. In an example, this bit is present in the first flit ofall packet types. DID 8 Destination NOC Endpoint ID bits <7:0> STAT 4Response Status PATH 8 The PATH field used to specify a path through aCPI TID fabric to force ordering between packets. For both CPI nativeand AXI over CPI, the read response packet's PATH field can contain atransaction identifier (TID) value. CP 2 Credit Present/Path Ordering.The CP field contains an encoded value that specifies both whether thefield CR of flits 3-N of the packet contains credit return informationand whether path ordering is enabled. Lines 2 and beyond DATA 32 ReadResponse Data, bits N*8-1:0 CR/RSV 4 Credit Return Information RSV 4Reserved

FIG. 7 is a flow chart of an example of a method 700 for managinghazards in a memory controller, according to an embodiment. Operationsof the method 700 are performed by computer hardware, such as thatdescribed with respect to FIG. 1 (e.g., the memory controller chiplet140), FIG. 2 , FIG. 3 , or FIG. 8 (e.g., processing circuitry).

At operation 705, a memory request that includes a base memory addressis received. In an example, the memory request is for data in a memorymanaged by a memory controller. In an example, the memory request is fora computation component, such as a cryptographic unit, a built-in atomicunit, or a PAU. In an example, the memory request is in the form of aCPI request, such as the CPI request 500 of FIG. 5 .

At operation 710, an index is computed from the base memory address. Inan example, the index is computed from the base memory address byhashing the base memory address.

At operation 715, a lookup is performed using the index to find a lock.In an example, the lock is represented by a single bit. In an example,to perform the lookup using the index, an entry in data structure thatcorresponds to the index is obtained. In an example, the data structureis an array. In an example, the data structure has fewer entries thanbase memory addresses in an addressable space for the memory controller.

At operation 720, the memory request is stored in a buffer in responseto finding the lock. In an example, the buffer is one of severalbuffers. Here, the buffer to which the memory request is stored is theonly one of the several buffers that corresponds to the lock and theother buffers respectively correspond to other possible locks. Thus,each buffer uniquely corresponds to one possible lock, linking thebuffers and the possible locks.

At operation 725, the memory request is removed from the buffer inresponse to a signal to clear the lock. In an example, removing thememory request from the buffer includes selecting the buffer frommultiple of the several buffers by an arbiter. In an example, selectionsof the arbiter are placed in a queue. In an example, the memory requestis removed from the queue in response to an absence of a memory requestthat had no lock.

At operation 730, performing the memory request after removal from thebuffer. In an example, performing the memory request includes settingthe lock

FIG. 8 illustrates a block diagram of an example machine 800 with which,in which, or by which any one or more of the techniques (e.g.,methodologies) discussed herein can be implemented. Examples, asdescribed herein, can include, or can operate by, logic or a number ofcomponents, or mechanisms in the machine 800. Circuitry (e.g.,processing circuitry) is a collection of circuits implemented intangible entities of the machine 800 that include hardware (e.g., simplecircuits, gates, logic, etc.). Circuitry membership can be flexible overtime. Circuitries include members that can, alone or in combination,perform specified operations when operating. In an example, hardware ofthe circuitry can be immutably designed to carry out a specificoperation (e.g., hardwired). In an example, the hardware of thecircuitry can include variably connected physical components (e.g.,execution units, transistors, simple circuits, etc.) including a machinereadable medium physically modified (e.g., magnetically, electrically,moveable placement of invariant massed particles, etc.) to encodeinstructions of the specific operation. In connecting the physicalcomponents, the underlying electrical properties of a hardwareconstituent are changed, for example, from an insulator to a conductoror vice versa. The instructions enable embedded hardware (e.g., theexecution units or a loading mechanism) to create members of thecircuitry in hardware via the variable connections to carry out portionsof the specific operation when in operation. Accordingly, in an example,the machine readable medium elements are part of the circuitry or arecommunicatively coupled to the other components of the circuitry whenthe device is operating. In an example, any of the physical componentscan be used in more than one member of more than one circuitry. Forexample, under operation, execution units can be used in a first circuitof a first circuitry at one point in time and reused by a second circuitin the first circuitry, or by a third circuit in a second circuitry at adifferent time. Additional examples of these components with respect tothe machine 800 follow.

In alternative embodiments, the machine 800 can operate as a standalonedevice or can be connected (e.g., networked) to other machines. In anetworked deployment, the machine 800 can operate in the capacity of aserver machine, a client machine, or both in server-client networkenvironments. In an example, the machine 800 can act as a peer machinein peer-to-peer (P2P) (or other distributed) network environment. Themachine 800 can be a personal computer (PC), a tablet PC, a set-top box(STB), a personal digital assistant (PDA), a mobile telephone, a webappliance, a network router, switch or bridge, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein, such as cloud computing, software as aservice (SaaS), other computer cluster configurations.

The machine (e.g., computer system) 800 can include a hardware processor802 (e.g., a central processing unit (CPU), a graphics processing unit(GPU), a hardware processor core, or any combination thereof), a mainmemory 804, a static memory (e.g., memory or storage for firmware,microcode, a basic-input-output (BIOS), unified extensible firmwareinterface (UEFI), etc.) 806, and mass storage 808 (e.g., hard drives,tape drives, flash storage, or other block devices) some or all of whichcan communicate with each other via an interlink (e.g., bus) 830. Themachine 800 can further include a display unit 810, an alphanumericinput device 812 (e.g., a keyboard), and a user interface (UI)navigation device 814 (e.g., a mouse). In an example, the display unit810, input device 812 and UI navigation device 814 can be a touch screendisplay. The machine 800 can additionally include a storage device(e.g., drive unit) 808, a signal generation device 818 (e.g., aspeaker), a network interface device 820, and one or more sensors 816,such as a global positioning system (GPS) sensor, compass,accelerometer, or other sensor. The machine 800 can include an outputcontroller 828, such as a serial (e.g., universal serial bus (USB),parallel, or other wired or wireless (e.g., infrared (IR), near fieldcommunication (NFC), etc.) connection to communicate or control one ormore peripheral devices (e.g., a printer, card reader, etc.).

Registers of the processor 802, the main memory 804, the static memory806, or the mass storage 808 can be, or include, a machine readablemedium 822 on which is stored one or more sets of data structures orinstructions 824 (e.g., software) embodying or utilized by any one ormore of the techniques or functions described herein. The instructions824 can also reside, completely or at least partially, within any ofregisters of the processor 802, the main memory 804, the static memory806, or the mass storage 808 during execution thereof by the machine800. In an example, one or any combination of the hardware processor802, the main memory 804, the static memory 806, or the mass storage 808can constitute the machine readable media 822. While the machinereadable medium 822 is illustrated as a single medium, the term “machinereadable medium” can include a single medium or multiple media (e.g., acentralized or distributed database, or associated caches and servers)configured to store the one or more instructions 824.

The term “machine readable medium” can include any medium that iscapable of storing, encoding, or carrying instructions for execution bythe machine 800 and that cause the machine 800 to perform any one ormore of the techniques of the present disclosure, or that is capable ofstoring, encoding or carrying data structures used by or associated withsuch instructions. Non-limiting machine readable medium examples caninclude solid-state memories, optical media, magnetic media, and signals(e.g., radio frequency signals, other photon based signals, soundsignals, etc.). In an example, a non-transitory machine readable mediumcomprises a machine readable medium with a plurality of particles havinginvariant (e.g., rest) mass, and thus are compositions of matter.Accordingly, non-transitory machine-readable media are machine readablemedia that do not include transitory propagating signals. Specificexamples of non-transitory machine readable media can include:non-volatile memory, such as semiconductor memory devices (e.g.,electrically programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM)) and flash memorydevices; magnetic disks, such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

In an example, information stored or otherwise provided on the machinereadable medium 822 can be representative of the instructions 824, suchas instructions 824 themselves or a format from which the instructions824 can be derived. This format from which the instructions 824 can bederived can include source code, encoded instructions (e.g., incompressed or encrypted form), packaged instructions (e.g., split intomultiple packages), or the like. The information representative of theinstructions 824 in the machine readable medium 822 can be processed byprocessing circuitry into the instructions to implement any of theoperations discussed herein. For example, deriving the instructions 824from the information (e.g., processing by the processing circuitry) caninclude: compiling (e.g., from source code, object code, etc.),interpreting, loading, organizing (e.g., dynamically or staticallylinking), encoding, decoding, encrypting, unencrypting, packaging,unpackaging, or otherwise manipulating the information into theinstructions 824.

In an example, the derivation of the instructions 824 can includeassembly, compilation, or interpretation of the information (e.g., bythe processing circuitry) to create the instructions 824 from someintermediate or preprocessed format provided by the machine readablemedium 822. The information, when provided in multiple parts, can becombined, unpacked, and modified to create the instructions 824. Forexample, the information can be in multiple compressed source codepackages (or object code, or binary executable code, etc.) on one orseveral remote servers. The source code packages can be encrypted whenin transit over a network and decrypted, uncompressed, assembled (e.g.,linked) if necessary, and compiled or interpreted (e.g., into a library,stand-alone executable etc.) at a local machine, and executed by thelocal machine.

The instructions 824 can be further transmitted or received over acommunications network 826 using a transmission medium via the networkinterface device 820 utilizing any one of a number of transfer protocols(e.g., frame relay, internet protocol (IP), transmission controlprotocol (TCP), user datagram protocol (UDP), hypertext transferprotocol (HTTP), etc.). Example communication networks can include alocal area network (LAN), a wide area network (WAN), a packet datanetwork (e.g., the Internet), mobile telephone networks (e.g., cellularnetworks), plain old telephone (POTS) networks, and wireless datanetworks (e.g., Institute of Electrical and Electronics Engineers (IEEE)802.11 family of standards known as Wi-Fi®, IEEE 802.16 family ofstandards known as WiMax®), IEEE 802.15.4 family of standards,peer-to-peer (P2P) networks, among others. In an example, the networkinterface device 820 can include one or more physical jacks (e.g.,Ethernet, coaxial, or phone jacks) or one or more antennas to connect tothe communications network 826. In an example, the network interfacedevice 820 can include a plurality of antennas to wirelessly communicateusing at least one of single-input multiple-output (SIMO),multiple-input multiple-output (MIMO), or multiple-input single-output(MISO) techniques. The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding orcarrying instructions for execution by the machine 800, and includesdigital or analog communications signals or other intangible medium tofacilitate communication of such software. A transmission medium is amachine readable medium.

Example 1 is a memory controller comprising: an interface configured toreceive a memory request that includes a base memory address; transformcircuitry configured to compute an index from the base memory address; alock table data structure configured to perform a lookup using the indexto find a lock; a buffer configured to store the memory request inresponse to finding the lock, the buffer being one of several buffers,the buffer being the only one of the several buffers that corresponds tothe lock, other buffers of the several buffers respectivelycorresponding to other possible locks; an arbiter configured to removethe memory request from the buffer in response to a signal to clear thelock; and a processing pipeline configured to perform the memory requestafter removal from the buffer.

In Example 2, the subject matter of Example 1, wherein, to compute theindex from the base memory address, the transform circuitry isconfigured to hash the base memory address.

In Example 3, the subject matter of Example 2, wherein, to perform thelookup using the index, the lock table data structure returns an entryin an array that corresponds to the index.

In Example 4, the subject matter of any of Examples 2-3, wherein thelock table data structure has fewer entries than base memory addressesin an addressable space for the memory controller.

In Example 5, the subject matter of any of Examples 2-4, wherein thelock is represented by a single bit.

In Example 6, the subject matter of any of Examples 1-5, wherein, toremove the memory request from the buffer, the arbiter is configured toselect the buffer from multiple of the several buffers.

In Example 7, the subject matter of Example 6, wherein selections of thearbiter are placed in a queue.

In Example 8, the subject matter of Example 7, wherein the memoryrequest is removed from the queue in response to an absence of a memoryrequest that had no lock.

In Example 9, the subject matter of any of Examples 1-8, wherein, toperform the memory request, the processing pipeline is configured to setthe lock.

In Example 10, the subject matter of any of Examples 1-9, wherein thememory controller is a chiplet in a chiplet system.

Example 11 is a method comprising: receiving, at a memory controller, amemory request that includes a base memory address; computing an indexfrom the base memory address; performing a lookup using the index tofind a lock; storing the memory request in a buffer in response tofinding the lock, the buffer being one of several buffers, the bufferbeing the only one of the several buffers that corresponds to the lock,other buffers of the several buffers respectively corresponding to otherpossible locks; removing the memory request from the buffer in responseto a signal to clear the lock; and performing the memory request afterremoval from the buffer.

In Example 12, the subject matter of Example 11, wherein computing theindex from the base memory address includes hashing the base memoryaddress.

In Example 13, the subject matter of Example 12, wherein performing thelookup using the index includes obtaining an entry in an array thatcorresponds to the index.

In Example 14, the subject matter of any of Examples 12-13, wherein thelookup is performed on a data structure that has fewer entries than basememory addresses in an addressable space for the memory controller.

In Example 15, the subject matter of any of Examples 12-14, wherein thelock is represented by a single bit.

In Example 16, the subject matter of any of Examples 11-15, whereinremoving the memory request from the buffer includes selecting thebuffer from multiple of the several buffers by an arbiter.

In Example 17, the subject matter of Example 16, wherein selections ofthe arbiter are placed in a queue.

In Example 18, the subject matter of Example 17, wherein the memoryrequest is removed from the queue in response to an absence of a memoryrequest that had no lock.

In Example 19, the subject matter of any of Examples 11-18, whereinperforming the memory request includes setting the lock.

In Example 20, the subject matter of any of Examples 11-19, wherein thememory controller is a chiplet in a chiplet system.

Example 21 is a machine-readable medium including instructions that,when executed by circuitry of a memory controller, cause the memorycontroller to perform operations: receiving a memory request thatincludes a base memory address; computing an index from the base memoryaddress; performing a lookup using the index to find a lock; storing thememory request in a buffer in response to finding the lock, the bufferbeing one of several buffers, the buffer being the only one of theseveral buffers that corresponds to the lock, other buffers of theseveral buffers respectively corresponding to other possible locks;removing the memory request from the buffer in response to a signal toclear the lock; and performing the memory request after removal from thebuffer.

In Example 22, the subject matter of Example 21, wherein computing theindex from the base memory address includes hashing the base memoryaddress.

In Example 23, the subject matter of Example 22, wherein performing thelookup using the index includes obtaining an entry in an array thatcorresponds to the index.

In Example 24, the subject matter of any of Examples 22-23, wherein thelookup is performed on a data structure that has fewer entries than basememory addresses in an addressable space for the memory controller.

In Example 25, the subject matter of any of Examples 22-24, wherein thelock is represented by a single bit.

In Example 26, the subject matter of any of Examples 21-25, whereinremoving the memory request from the buffer includes selecting thebuffer from multiple of the several buffers by an arbiter.

In Example 27, the subject matter of Example 26, wherein selections ofthe arbiter are placed in a queue.

In Example 28, the subject matter of Example 27, wherein the memoryrequest is removed from the queue in response to an absence of a memoryrequest that had no lock.

In Example 29, the subject matter of any of Examples 21-28, whereinperforming the memory request includes setting the lock.

In Example 30, the subject matter of any of Examples 21-29, wherein thememory controller is a chiplet in a chiplet system.

Example 31 is a system comprising: means for receiving, at a memorycontroller, a memory request that includes a base memory address; meansfor computing an index from the base memory address; means for means forstoring the memory request in a buffer in response to finding the lock,the buffer being one of several buffers, the buffer being the only oneof the several buffers that corresponds to the lock, other buffers ofthe several buffers respectively corresponding to other possible locks;means for removing the memory request from the buffer in response to asignal to clear the lock; and means for performing the memory requestafter removal from the buffer.

In Example 32, the subject matter of Example 31, wherein the means forcomputing the index from the base memory address include means forhashing the base memory address.

In Example 33, the subject matter of Example 32, wherein the means forperforming the lookup using the index include means for obtaining anentry in an array that corresponds to the index.

In Example 34, the subject matter of any of Examples 32-33, wherein thelookup is performed on a data structure that has fewer entries than basememory addresses in an addressable space for the memory controller.

In Example 35, the subject matter of any of Examples 32-34, wherein thelock is represented by a single bit.

In Example 36, the subject matter of any of Examples 31-35, wherein themeans for removing the memory request from the buffer include means forselecting the buffer from multiple of the several buffers by an arbiter.

In Example 37, the subject matter of Example 36, wherein selections ofthe arbiter are placed in a queue.

In Example 38, the subject matter of Example 37, wherein the memoryrequest is removed from the queue in response to an absence of a memoryrequest that had no lock.

In Example 39, the subject matter of any of Examples 31-38, wherein themeans for performing the memory request include means for setting thelock.

In Example 40, the subject matter of any of Examples 31-39, wherein thememory controller is a chiplet in a chiplet system.

Example 41 is at least one machine-readable medium includinginstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform operations to implement of any ofExamples 1-40.

Example 42 is an apparatus comprising means to implement of any ofExamples 1-40.

Example 43 is a system to implement of any of Examples 1-40.

Example 44 is a method to implement of any of Examples 1-40.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments in which theinvention can be practiced. These embodiments are also referred toherein as “examples”. Such examples can include elements in addition tothose shown or described. However, the present inventors alsocontemplate examples in which only those elements shown or described areprovided. Moreover, the present inventors also contemplate examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” can include “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein”. Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to impose numerical requirements on their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) can be used in combination with each other. Otherembodiments can be used, such as by one of ordinary skill in the artupon reviewing the above description. It is submitted with theunderstanding that it will not be used to interpret or limit the scopeor meaning of the claims. Also, in the above Detailed Description,various features can be grouped together to streamline the disclosure.This should not be interpreted as intending that an unclaimed disclosedfeature is essential to any claim. Rather, inventive subject matter canlie in less than all features of a particular disclosed embodiment.Thus, the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separateembodiment, and it is contemplated that such embodiments can be combinedwith each other in various combinations or permutations. The scope ofthe invention should be determined with reference to the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

What is claimed is:
 1. A chiplet memory controller comprising: firststorage configured to hold a lock data structure; second storageconfigured to hold multiple buffers; and processing circuitry configuredto store a memory request in a buffer of the multiple buffers based on alock in the data structure being set, the lock being one of severallocks, the lock corresponding to the memory request based on a featureof the memory request, the buffer corresponding to the lock.
 2. Thechiplet memory controller of claim 1, wherein the chiplet memorycontroller is one of multiple chiplets in a chiplet system.
 3. Thechiplet memory controller of claim 2, wherein the memory request wassent by a second chiplet of the chiplet system.
 4. The chiplet memorycontroller of claim 3, wherein the memory request was received over apacket-based chiplet network.
 5. The chiplet memory controller of claim4, wherein the memory request is directed to a memory location in amemory managed by the chiplet memory controller, the memory connected tothe chiplet memory controller via a memory bus.
 6. The chiplet memorycontroller of claim 1, wherein the feature of the memory request is ahash of the memory address.
 7. The chiplet memory controller of claim 6,wherein multiple memory addresses hash to a same entry in the datastructure.
 8. The chiplet memory controller of claim 1, wherein theprocessing circuitry is configured to arbitrate memory requests byremoving the memory request from the buffer based on a clearing of thelock.
 9. The chiplet memory controller of claim 1, wherein theprocessing circuitry is configured to process the memory request basedon removing the memory request from the buffer.
 10. The chiplet memorycontroller of claim 9, wherein the processing circuitry is configured toset the lock as part of processing the memory request.
 11. A methodcomprising: establishing a lock data structure in a first storage of achiplet memory controller; establishing multiple buffers in a secondstorage of the chiplet memory controller; and storing, by processingcircuitry of the chiplet memory controller, a memory request in a bufferof the multiple buffers based on a lock in the data structure being set,the lock being one of several locks, the lock corresponding to thememory request based on a feature of the memory request, the buffercorresponding to the lock.
 12. The method of claim 11, wherein thechiplet memory controller is one of multiple chiplets in a chipletsystem.
 13. The method of claim 12, comprising receiving, at the chipletmemory controller, the memory request from a second chiplet of thechiplet system.
 14. The method of claim 13, wherein the memory requestwas received over a packet-based chiplet network.
 15. The method ofclaim 14, wherein the memory request is directed to a memory location ina memory managed by the chiplet memory controller, the memory connectedto the chiplet memory controller via a memory bus.
 16. The method ofclaim 11, wherein the feature of the memory request is a hash of thememory address.
 17. The method of claim 16, wherein multiple memoryaddresses hash to a same entry in the data structure.
 18. The method ofclaim 11, comprising arbitrating, by the processing circuitry, memoryrequests by removing the memory request from the buffer based on aclearing of the lock.
 19. The method of claim 11, comprising processing,by the processing circuitry, the memory request based on removing thememory request from the buffer.
 20. The method of claim 19, whereinprocessing the memory request includes setting the lock as part ofprocessing the memory request.